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ABSTRACT 

Results of a study of a pilot program exploring the 
application of reflective practice to educational administration are 
reported in this paper. The program goal was to create a design 
studio for the development of reflective practice among educational 
administration students. To accomplish this goal, the design studio 
provided explicit instruction and experience in applying theoretical, 
empirical, and experiential knowledge to practical problems. Expert 
practitioners coached students through the processes of problem 
solving and definition as they developed action plans. This process 
of cognitive coaching focused on school problems rather than on 
traditional subjects of study. Six coaches who were practicing school 
administrators were selected from the faculty of the Department of 
Educational Administration at the University of Utah to supervise 
graduate students enrolled in a seminar on the principalship. Program 
evaluation involved document analysis, participant observation, exit 
interviews with students and coaches, and review panel assessments. 
Findings demonstrate the need for coaches' trii.-<ning and practice in 
reflective and inductive questioning, fixed prerequisites for a 
studant-focused design studio, continuous program evaluation, and 
provision of specific learning cases. (30 references) (LMI) 
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LEARNING TO LEAD: 
REFLECTIVE PRACTICE IN PRESERVICE EDUCATION 

Ann Weaver Hart 
Nancy B. Sorensen 
Kerrie Naylor 

University of Utah 

Because the problems of public education in the United States are 
becoming more complex and varied as the century draws to a close, new 
approaches to defining and solving school system problems are more important 
to educational administrators. School restructuring, site-based management, 
shared governance, and merit incentives are among the reforms being used to 
address these educational problems. The school leaders who will make these 
new ways of organizing schools work need to acquire problem-solving skills. 
These skills will enable them to apply knowledge from many sources to the 
actions they take in schools. One approach to improving the problem- solving 
ability of school administrators (and other professionals) is known as 
''reflective practice.'' Interest in this approach to school leadership is 
growing, but specific programs for nurturing these skills during the formal 
education of administrators are just being developed (Hart, 1986, 1990a, 
1990b: Hart & Sorensen, 1989; Murphy, 1990). 

In this paper, we report the results of a study of a pilot program 
exploring the applications of reflective practice to educational 
administration education. The scudy investigated the power and potential of 
reflection as the principle around which a prer ^r/ice course in educational 
administration could be designed. It also provided rn evaluation of 
procedures used to accomplish this task. Modeled after the design studio 
proposed by Schou (1987), this pilot program used expert practitioners to 
coach students through the process of thinking about problem definition and 
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problem- solving as they develop plans for action. This process of cognitive 
coaching focused on school problems (problem-based) rather than traditional 
subjects of study (subject-based). The goal of the pilot program was to 
develop a design studio that could nurture reflective practice In students of 
educational administration as part of their formal course of study. To 
accomplish this goal, the design studio provided explicit instruction and 
experience in applying theoretical, empirical, and experiential knowledge to 
the problems of practice. 

The following sections of the paper Include a brief overview of the 
background of reflective practice and a description of the components of the 
pilot design studio, research methods, findings > and conclusions. 

Reflection: A Background 

Over the last few years » educators and practitioners in educational 
administration have explored a number of approaches to improving the 
application of research, theory, and experience to the practice of school 
administration (Hart, 1990b). A number of these exploratory approaches --one 
of which is problem-based, student- centered learning- -draw on traditions of 
cognitive science to improve problem identification and the creativity of 
solutions (Prestine, in press). A gradual shift In graduate professional 
training toward more problem-based learning is not unique to education. Other 
professional scho<^ds, working at the graduate level with adult students, are 
coming to rely on student -centered, problem-based methods of instruction in 
order to prepare expert practitioners highly skilled in applying the knowledge 
base of their fields to the problems of practice (Ramsey & Whitman, 1989). 
Focusing on the thinking process of students, these methods of instruction 
produce subject mastery equivalent to that achieved in rigorous subject -based 



instruction. They also produce superior application, transfer, and problem- 
solving (Barrows, 1988). Consequently, problem-based learning which is 
focused on students rather than subject-based instruction which is focused on 
the instructor (Ramsey & Whitman, 1989) is gaining increased attention in 
professional schools- These methods of instruction have been developed in 
response to the ubiquitous and jarring experience students describe when they 
move from formal academic study into the practice of their professions and 
find that they have not developed strategies and skills in applying what they 
know to what they do. 

Reflection (as it is used here) is much more than quiet thinking over 
past events. It aims toward a goal such as a set of solutions to dilemmas or 
problems or the redefining and understanding of "the problem." In pursuit of 
the goal, the person engaged in reflection creates a sequence of ideas, 
projecting possible consequences that will result in an outcome or in a series 
of events (Dewey, 1933). Because a knowledge base is required in order to 
form accurate projections, knowledge is a critical component of this process. 
This linkage between knowledge and potential outcomes, the progression of 
thought that links and expands complex parts of the whole, can lead to 
increased complexity, creativity, and surprise as new ideas emerge and lead to 
unexpected but productive conclusions. This outcome of reflection may explain 
why students in professional schools where the process is being deliberately 
applied have been shown to exhibit superior application, transfer, and 
problem-solving outcomes. 

Reflection illustrates the application of cognition theories to 
professional education, providing a theoretical framework for understanding 
the process through which application, transfer, and problem- solving might be 



enhanced. Through ♦'he use of mental advanced organizers » abstract patterns of 
reality called schema, people form initial perceptions and judgments of new 
events* A schema is a pattern of concepts and associations that repeatedly 
occur together which is held within the mind of a person and forms the 
expectations through which the person interprets new, related information 
(Thorndyke & Hayes-Roth, 1979). Schemata help people process and transfer 
knowledge from one situation to another (Luiten & Amos, 1980), and reflection 
provides practice in deliberately and systematically recognizing similar 
patterns in unique individual events. This can occur because relevant 
deviations from habitual behavior create vivid memories (Bower, Black, & 
Turner, 1979). These memories become resources for future reflection and 
action, and a relationship between action and memory is established. 

The quality of future reflection and action also is increased as skill 
and knowledge increase. This occurs in two ways. First, as learning is 
assisted and inappropriate inferences reduced, skill improves. Second, as 
knowledge increases, the ability to draw appropriate inferences improves 
(Pearson, Hansen, & Gordon, 1979; Nickolson & Imlack, 1981). Intransigent 
problems and surprises that challenge existing schemata bring with them an 
important addition to the quality of reflection. As Schon (1983, p. 62-63) 
pointed out: When the phenomenon at hand eludef^ the ordinary categories of 
knowledge -in-pract ice , presenting itself as unique or unstable, the 
practitioner may surface and criticize his initial understanding of the 
phenomenon, construct a new description of it, and test the new description by 
an on-the-spot experiment. 
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The new description becomes an adjustment in a schema which will, in turn, be 
applied the next time an experience with similar (and some unique) 
characteristics occurs . 

The final step in reflective practice is action. Action follows after a 
schema is applied, adjustments made when necessary, and inferences drawn. 
When steps are taken, the act is integrated with knowledge and thought. 
Outcomes of action may lead to further adjustments in the existing schema. 
Continuous repetitions of this process throughout a professional career lead 
to development as a reflective practitioner. The logic supporting a move 
toward the development and inclusion of experiences designed to teach the 
reflective process in educational administration education is thus supported 
by cognition theory, philosophy, and research on the application of 
professional knowledge by practitioners who have been educated using 
traditional subject-based, instructor focused methods and those who have been 
educated using more problem-based, student focused methods. 

Components of the Pilot Design Studio 

The conceptual framework for a design studio was developed by members of 
the Department of Educational Administration at the University of Utah as part 
of a Danforth Foundation project to improve the quality of university programs 
of educational administration. First, the faculty participated in initial 
planning based on a literature review, department discussion, feedback from 
other universities in the Danforth project, and program review. Then faculty 
members and graduate students met to form a preliminary outline for a pilot 
structure for the design studio. These discussions led to a commitment to run 
a trial studio during the winter of 1990. Final planning was completed in the 
summer and fall of 1989 (Hart & Sorensen, 1989). 



Components of the studio included: (1) problem-based stimulus materials 
for students; (2) professional coaches; (3) theoretical/empirical resources; 
(4) student action plans (written); (5) and a panel of professors, 
superintendents, and principals who reviewed written plans and oral arguments, 
questioned students, and prepared systematic feedback and assessment. 

Stimulus materials were prepared by the faculty and a graduate student 
from a core of case problems collected over a four year period in a seminar on 
the principalship, published cases, and faculty and graduate student 
experiences. Two sets of materials, one for elementary school and one for 
secondary school, were written. Five major dilemmas were presented in both 
situations. The dilemmas included issues relevant to traditional courses and 
seminars in administration and leadership, policy, personnel, finance, law, 
curriculxan, and instruction. While drawn from real field experiences, the 
cases were fictionalized in order to present the range and depth of the 
administrative preparation curriculum (Ramsey & Whitman, 1989). Prior to the 
beginning of the design studio, the coaches read each set of stimulus 
materials and suggested some changes and additions. A teacher supervision 
issue was added to the secondary school issues as a result of this review. 
Students chose to work from either the elementary or secondary school 
materials. 

Coaches were selected from adjunct and clinical faculty of the 
Department of Educational Administration at the University of Utah who were 
practicing school administrators. Six coaches agreed to participate. They 
included elementary, junior high, and high school principals and assistant 
principals. One of the coaches was the president-elect of the national 
Association for Supervision and Curriculum Development (ASCD) , Coaches met 



prior to the studio sessions for orientation and training in reflective 
questioning, coaching, and problem- solving techniques. The faculty member 
responsible for the seminar and design studio served as a seventh coach during 
studio sessions* 

Background reading for participants (students and coaches) presented 
some challenge for the operation of the pilot studio* A full problem-based 
course of study would move students systematically through problems designed 
to send students to a curriculum and to resource materials necessary to 
address the assigned problems so that subject mastery would be complete. At 
this experimental stage, this was not possible. Because students were drawn 
from those enrolled in a graduate seminar on the principalship , prerequisites 
could not be required for the pilot. While most of the students had completed 
a substantial proportion of their course work for administrative 
certification, two of the five who eventually completed the studio had not* 
Two basic texts thus were assigned as a common knowledge base from which 
students would draw (Bolman & Deal, 1984; Duke, 1987)* 

Studio activities covered a ten week period, primarily because of the 
limitations imposed by the academic quarter at the University of Utah* The 
first four weeks of the seminar, students systematized their notes and 
readings from previous course work and together completed the core assigned 
readings in school administration (Duke, 1987), organizational analysis 
(Bolman & Deal, 1984), and reflective practice (Hart, 1990a). Students kept 
weekly journals of their readings and seminar discussions* During the next 
four weeks students met for 1 1/2 hours with coaches in refleCwive questioning 
sessions followed by 1 1/2 hour debriefing sessions with the instructor. 




Coaches rotated among students so that each coach and each student would have 
at least two sessions together if possible. 

At the end of the four weeks with the coaches, each student prepared a 
written action plan which included the following sections: (1) a philosophy 
or vision statement that included a description of the hypothetical school 
under their leadership; (2) specific action recommendations for each of the 
five dilemmas in the stimulus materials; and (3) an overall plan for the 
school which unified their action plans across dilemmas. Written plans 
included specific references to empirical and theoretical knowledge acquired 
during their formal studies (in this or any other course), individual 
experiences and coachos' feedback, and exemplary practice collected as part of 
their data gathering and problem-solving. Action plans then were submitted to 
a panel of professors, superintendents, and principals who read and evaluated 
them in advance. The panel sat as a review board for oral presentations and 
questions at the end of the 10th week of the quarter. The panel members and 
instructor provided feedback to students in written and oral form. 

Research Methods 

Three researchers participated in the studio design, writing of 
materials, orientation of coaches, data collection and data analysis-«*one 
faculty member and two graduate students in the Department of Educational 
Administration at the University of Utah. 

Data Collection 

Data included: student journals; participant observation notes 
(collected as systematic field notes by the instructor who also worked as a 
coach and a review panel member) ; systematic field notes of all coaching 
sessions collected by a second researcher; stimulus materials; orientation 
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materials for participants; orientation materials for coaches; exit Interviews 
of students and coaches; final action plans; and review panel assessments of 
action plans and field notes of feedback given by the review panel members to 
the students. 

The first source of data was student journals. Each student was 
required to keep a journal in which they reacted to the assigned readings 
during the first four weeks of the quarter and reflected on their problem- 
based experience during the subsequent six weeks. The journals provided data 
about the accessibility and usefulness of the reading material chosen for the 
course and a glimpse into the struggles and insights students experienced. 

The instructor (and first researcher) provided a second source of data. 
In addition to organizing the pilot, the instructor worked as a cognitive 
coach during the four coaching sessions. Following each session and after the 
review panel presentations, the instructor wrote a description of the events 
of the session and a reflection on the experience in the form of systematic 
field notes . 

Third, the second researcher attended each session of the design studio. 

« 

During each coaching session, acting as a nonparticlpant observer, she 
collected scripts of the interaction between coaches and students. Following 
each session, these data were expanded into systematic field notes in three 
sections: (a) a verbatim account of segments of the exchanges between coaches 
and students and summaries of other exchanges, descriptions of behavior, a 
record of periods of silence and data scanning by participants, other 
objective details of the session, and a record of time elapsed; (b) a 
description of the affective characteristics of each observed session between 
coach and student; and (c) a general description of the session. A systematic 
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sampling of coaches and students was made so that each coach and each student 
was observed In varying pair combinations. Coaches each attended two of four 
sessions and were observed at least twice (with one exception) . Students were 
observed three or four times (with one exception, a student who was absent 
from two of the four sessions) . 

Fourth, the documents prepared for students and coaches were a source of 
data to be analyzed for relevance and usefulness Uo the overall experience In 
the design studio. These Included the stimulus materials or problem 
descriptions, the orientation materials for students, and the orientation 
materials for coaches. 

Following the presentation of the final action plans to the review 
panel, a researcher (not the Instructor) Interviewed the coaches and students. 
Participants were asked to assess the materials, process, and products n£ the 
design studio, describe the most useful and most difficult or negative aspects 
of the experience, and make suggestions for Improving the design studio In the 
future. Interviews were conducted at the students' and coaches' schools or at 
the university at the convenience of the participant. Field notes from these 
Interviews, which lasted from 30 minutes to one hour and were completed within 
one week of the final presentations, provided a fifth source of data for the 
study. 

The final action plans prepared by students provided a sixth source of 
data. The students chose to impose an organizational pattern for the final 
action plans to assure that consistent expectations and standards were applied 
to the assessment of each plan. These sections were: (1) a philosophy or 
vision statement that included a description of the hypothetical school under 
their leadership; (2) specific action recommendations for each of the five 
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dilenunas in the stimulus materials; and (3) an overall plan for the school 
which unified their action recommendations across dilemmas. Students 
explicitly cited research and practice to support their recommendations. 

Finally, each member of the review panel prepared written comments on 
the action plans and then participated in a formal presentation and 
questioning session. Final evaluation of the action plans came from written 
comments of panel members and systematic field notes taken by a researcher on 
the feedback given by the review panel members to the students during the 
presentations p field notes made by the instructor during and after the panel 
review presentations , and a final reading by the instructor. 

Data Analysis 

Data analysis followed established procedures of qualitative research. 
Conceptual categories were used to code data. Then the researchers prepared 
data summary sheets, analytical matrices, and text analysis summaries (Cuba & 
Lincoln, 1983; Miles & Huberman, 1984). For this preliminary report, the 
observation notes, exit interviews, action plans, and action plan assessments 
provided the primary sources of data. 

First, coding categories based on problem-based professional education, 
reflection, and cognition research were developed from the original research 
framework. These referred to common problem- solving errors made by students, 
inferential questionings and the common readings. 

Second, data were read by one of the researchers and a set of 
preliminary inductive categories developed. These were then used in concert 
with the deductive categories to code the observation notes and exit 
interviews in a second reading. A second researcher then conducted a coding 
audit, noting differences or omissions from the first coding and suggesting 



additional categories that the preliminary coding neglected to encompass. A 
second coding audit was then conducted by a third researcher. Where 
disagreements remained, the two auditors met and agreed on the final coding. 
This process led to the development of the final categories discussed in the 
findings section of this paper. 

Third, the researchers created data summary sheets and matrices of the 
final coding. The summary sheets included coding totals, across time, and by 
participant. The matrices presented data for participants (coaches and 
students) by coding frequency, participants by session, and session totals 
(providing a view of the development of issues across time) . This process 
provided a means to Assess the prevalence of particular issues and their 
frequency in relation to individual participants over time. 

Finally, the researchers compared summary sheets, matrices, and raw data 
coding to the assessments of the final action plans and presentations. This 
process provided a mecns of comparing process and outcome data. 

Findings 

The findings related to coaches and students are reported separately for 
ease of discussion and are organized by conceptual category. Issues of 
frequency, intensity, preference, and effectiveness are addressed within each 
section. 

Coaches 

The coaches differed in their ability to stimulate students^ thinking 
and in the usefulness of their preferred approaches from session one to 
session four. They also provided helpful suggestions for future studio 
structures. The coach identified by students as least helpful overall was the 
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most directive about what should be done from the first session, asked the 
fewest questions 9 and provided no conceptual or research guidance. 
Design Studio Processes 

The conversation between coaches and students fell Into two main 
categories-* casual talk, either job related or personal, and problem focused 
talk, A third category of talk focused on the process of the cognitive 
coaching sessions and occurred with about half the frequency. 
Casual Conversation 

The benefit of personal Interaction, aside from the benefits of problem- 
solving, surprised the Instructor and studio participants. Coaches told 
stories about their schools that were similar to the situations In the 
stimulus materials. While the coaches kept story telling to a minimum (only 
thirteen stories among six coaches In four sessions), the students said It 
helped them get to know the coaches. It also gave them an Increased ser.se of 
belonging In the profession of educational administration. The next most 
frequent form of conversation in which coaches and students engaged was 
related to educational Issues In the state and their communities. Four of the 
six coaches used this small talk to break the Ice, Students, too, sometimes 
Initiated these conversations but less frequently than did coaches. However, 
the two students whose flna?. action plans received the highest ratings from 
panel members and the Instructor were the least likely to Initiate 
conversation on personal and community topics. 

Another form of casual exchange, often as the sessions were just getting 
started, was job talk. One coach referred a student to a specific job 
opening. Others talked In general about openings In their districts or about 
the major Issues they saw facing those entering administrative careers, 
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Finally, the coaches used praise to dir^ - Cudents toward ideas they 
found promising. One coach said in session one, ^You have had four very good 
ideas.'' He then reflected back on the insights he felt the students had 
raised, reinforcing the early questioning. 
Problem Focused Talk 

Coaches used two techniques most frequently to address the issues raised 
in the stimulus materials: (a) suggesting possible actions and (b) asking for 
information from the student related directly to the stimulus materials. A 
third technique, conceptualizing or referring students directly to research or 
theory, was used less frequently (about half as often), but four of che six 
coache.^ directed students to helpful resources or raised conceptual issues. 
All three techniques, when used appropriately, were effective In helping 
students question their ideas. Coaches offered suggestions with equal 
frequency across all four sessions. In contrast, their use of questions 
designed to solicit information, brainstorm, or press students to expand their 
thinking increased from session one to session two and then decline rapidly. 
Coaches' use of questions to elicit information appeared a total of 60 times 
in the observation data. References by coaches to research were equal across 
sessions, except for a marked decline in session three (26 coded 
observations). However, only one of the four coaches who offered conceptual 
guidance or referred to research attended session three. 

Coaches offered suggestions in a number of useful ways and one way which 
was less useful. The useful suggestions directed students but stayed away 
from conclusions. For example, coaches said: "You might bring in a teacher 
you trust and observe together.'' "You need to check the district evaluation 
policy." "You protect yourself with due process." "If you get a policy from 
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the faculty council, there will be more support/' The coaches who relied on 
directives simply told the students what to do. One coach told a student to 
bring in "concerned parents who you could work with, and consider the parent's 
background and skills.'' Another time, the same coach said: 
You need to see how this fits into the overall school 
philosophy .you bring the information to a curriculum committee 
to see if it fits with the rest of the school. You as the 
principal don't want to confront the teachers with an ultimatum. 
You might have to do a holding nattern..., get it away from the 
emotional issues.... Put some articulate parents on the 
committee. 

Direct suggestions for possible action from coaches appeared 56 times in the 
observation data. 

By their own account, coaches found questioning more difficult to do 
than offering direct suggestions but managed Co use problem directed questions 
slightly more frequently that direct suggestions. Questions directed students 
toward district policy, challenged student categories, and probed the 
appropriaten-iss of studa .t pioblem-solving approaches. Coaches' requests for 
information that directed students toward problem-solving appeared 60 times in 
the data and were worded in a number of ways. For example: "How are these 
items inter-related?" "What do you really know about what is being taught [in 
the classroom]?" "How would you approach this?" "Can y^^ think of any other 
way to find out who is on duty?" "How can you get to the true feeling of 
teachers?" "What do you see as your role in this?" "Why not?" "Where would 
you go for information?" 
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The third technique coaches relied on directed students toward research, 
particular disciplines (such as law, organizational theory, or political 
science) , and conceptualizations of school administration. These statements 
and questions directing students toward specific knowledge sources were coded 
26 times. For example, coaches said: ''Think in terms of district policy and 
state law/' ''Look into the research on school climate." "Another problem is 
copyright law." "What does the effective schools literature say about 
curricular offerings?" "Have you thought about the consistency of the 
program- -the continuity?" "Have you thought about your philosophy?" "Have 
you checked her personnel file and made sure you understand all parts of the 
evaluation?" "What does the state law say about evaluation procedures?" 
Process Focused Talk 

The process Interaction between students and coaches focused on which 
issues in the stimulus materials students wished to take up during a session 
or took the form of negotiations about the process they would follow ("Let's 
talk a little about the readings."). At one point, an overwhelmed student 
said he was unsure what to say. Another said he was "realizing that the 
iceberg is bigger than I thought." 

Process talk between students and coaches declined steadily from session 
one to session four. In sessions one and two, ten such exchanges were coded; 
in session three, five were recorded; and in session four, only two. Even 
though the coaches attended only two of the four sessions and rotated through 
(a structure designed to minimize the demand on coaches), students and coaches 
quickly became more proficient in the process as time passed. 
Exit Interviews 
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In the exit Interviews, coaches; were asked a series of questions: (a) 
Could you give me a general reaction to your experience with the design studio 
as part of an educational administration program? (b) What direction did your 
questioning seem to take most comfortably? (c) What kind of questions yielded 
the best responses from students in your assessment? (d) What woi;ld you say 
might be the major advantages and disadvantages of an approach such as the 
design studio in educational administration programs? (e) What changes would 
you recommend? (f) What did you like most and least about the stimulus 
materials? Changes? and (g) Have you any feeling about this method's 
potential for inservice as well as preseirvice? 

Overall, coaches reacted positively to the design studio experience, 
calling it a "unique approach to training." They were "very enthusiastic," 
"very positive" and saw it as "very practical" and a "great opportunity to 
blend theory with the practical." Coaches agreed that its major advantage 
might be in the way the process "challenges preconceived ideas." Coaches also 
identified a number of problems with the design of the pilot. They pointed 
out that the stimulus materials combined with initial readings presented 
students with an "overwhelming amount of material," that "students seemed 
overwhelmed. " 

The structure of the studio received some criticism. Coaches said that, 
while the studio required a "large time commitment," the coaches probably 
should attend all four sessions rather than just two. To alleviate the 
pressure this would cause, they suggested that a stipend for the coaches would 
be justified. 

Coaches who were most effective (according to student reports) argued 
that they took their cues for questioning from the students and "always had to 
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come ba to looking at the full picture," asking ''how does this fit into the 
plan of the school- -the holistic approach.'' They ''tried to get students to 
see implications,'' to "look at what they would need to check out." Others 
reported that they felt "pressure from students to answer questions." 

When asked what kinds of questions yielded the "best" responses, coaches 
identified question forms that directed students toward problem solving. They 
said: "What if?" "Have you thought about...?" "Where would you find this 
information?" "How would you feel if...?" They also referred back to the 
need to look at the "whole perspective" and force students "to look at their 
own philosophies and beliefs." They felt that they needed to remind students 
to "look at facts." 

The advantages and disadvantages of the design studio in the eyes of 
coaches focused on high quality outcomes and high costs in time and overload 
resulting from the complexity of its demands. They argued that adding reading 
assignments at the front end was too much when followed directly by the 
problem-based phase of the studio. The advantages coaches saw centered in 
"the mix of theory and text book with the practical" under the guidance of the 
coaches, its "hands-on, experiential" nature, the variety of "mentors" the 
coaches represented and their varied perspectives, and the personal 
interaction between coaches and students. (One student volunteered after his 
exit interview that several administrators he had told about the design studio 
had said they would like to be coaches.) Coaches sometimes contradicted 
themselves when they listed disadvantages. Along with the "overwhelming" 
complexity of the materials and time restrictions, they mentioned that the 
design studio failed if it was meant to simulate the time frame under which 
decisions must be made in administrative work. One contended that "students 
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don't feel the time pressure'^ and the studio is ''not real life.'' One coach 
pointed out that he had a "tendency to give students too much information 
rather than get them to reflect," an observation that the field notes 
confirmed. 

The coaches had several suggestions for improving the structure of the 
studio. They agreed that more than two sessions involving each coach vould be 
helpful, that coaches should be more involved in the entire process- -meeting 
to discuss their work midway through the design studio, sitting in on the 
final action plan presentations, participating in short debriefing sessions 
like those held for students following each session, and meeting together at 
the end of the quarter for debriefing and planning. They also felt that the 
training and readings had not prepared them sufficiently and wanted more 
training, more work on questioning techniques, and role playing and practice 
sessions. 

Criticizing and praising the stimulus materials for the same 
characteristics- -(a) complexity that was almost overwhelming and (b) realism 
--coaches generally found them useful and well organized. They felt they were 
well written, "dealt with important points," and "dealt with practical and 
realistic issues." 

The potential of design studio work for administrator inservice 
development received mixed reviews. The coaches saw it as particularly useful 
for administrators in their first three years or so of work, as a means for 
dealing with common problems, and as a way to talk with other administrators 
and form professional networks. But one coach was adamant about "no writing!" 
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students 

Students, too, provided data from observations and interviews. Their 
action plans and debriefing sessions also provided important data for 
interpreting possible connections between process and outcomes. Because this 
is the first data set comparing process and outcomes, however, and because the 
number of participants was small, data do not justify drawing causal 
inferences . 

Design Studio Processes 

Students talked with coaches about interaction processes and engaged in 
three main problem- solving processes: (a) they productively pursued problem 
definition and solutions; (b) they committed problem- solving errors; and (c) 
they sought information and answers. 
Problem- solving 

Students effectively used relevant diagnostic questioning, synthesis, 
and brainstorming or hypothesizing in their interactions with coaches during 
the design studio sessions. They also openly expressed concern and 
uncertainty. Students most often used coaches' new concepts and theories 
after the sessions were over and talked about them during the debriefing 
sessions. Only four times did students use new concepts while still working 
with a coach to adjust the direction of their thinking. 

Relevant diagnostic projection or questioning appeared thirteen times in 
the observation field notes. Students seemed to use this approach most during 
the second session, but, like the use of concepts and theory by coaches 
described above, this apparent pattern may be a result of the observation 
sample during this session. The two most proficient problem- solvers (and 
those who produced the most highly rated action plans) were two of the three 
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students observed during this session. Students identified issues for future 
problems -solving such as "'communication networks,'' "problems between different 
groups of teachers/' district level involvement, questions about 
documentation, district services the could use (personnel, legal counsel), and 
research on discipline and reinforcement as means for pupil control. They 
questioned comparative criteria for reading programs, moved toward the 
examination of policy, asked "how specifically do you document" and "when do 
you write a letter of reprimand?" 

Students synthesized information appropriately 10 times while being 
observed. Incidents of synthesis increased with each session. Students used 
a variety of techniques to accomplish synthesis in their problem-solving 
(e.g., decision trees, summary sheets for issues and questions of policy, 
curriculum, instruction, participative decisionmaking, school climate). 

Brainstorming, hypothesizing, and seizing on serendipity as new 
directions unfolded during conversations with coaches became by far the 
students most common approach to problem- solving. This was observed 24 times. 
It was most familiar and elicited responses from coaches. Brainstorming 
dominated sessions one, two, and three as students began to define problems, 
got to know the coaches, and began to formulate preliminary conclusions. 
Students speculated about ways to "gather facts," projected possible teacher 
and parent responses to actions they might take ("They might say they don't 
have time; this might be an indication they have too many duties."), and "saw 
things from a whole different angle." During these exchanges, students 
sometimes credited coaches with helping them with a breakthrough. "I felt I 
was stopped, and now someone has opened the door." 
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Problem- solving Error.-; 

This section was ceded using categories derived from literature on 
problem-based professional education and common problem- solving errors that 
occur* Data were examined for the presence of pseudodiagnositicity (seeking 
data that will not be helpful), incorrect synthesis (unwarranted conclusions), 
inadequate synthesis (not coming to a conclusion that is warranted) , premature 
closure, and anchoring (selective lack of attention to important information 
inconsistent with earlier thinking or experience). Identifiable examples of 
these errors were most common during the first session, and no incidents of 
problem- solving errors appeared in the observation notes from the l.ist 
coaching session. However, we found these errors hard to identify from 
verbatim records and paraphrased conversation in the field notes. They were 
more identifiable when students made their final action recommendations in the 
action plans. 

Search for Unhelpful Information or Pseudodiagnositicity. Only one 
exchange between a student and coach was coded as an incident of unhelpful 
information search, and this occurred in the third session. Students were far 
more likely to reach unwarranted conclusions or limit the amount of 
information sought. 

In the action plans, this problem- solving error appeared as digressions 
from parsimonious solutions and elaborate plans only tangentially related to 
core issues. It also may have been apparent in two of the five plans that 
failed to formulate a persuasive case for a school-wide initiative that 
unified the approach to the five cases v/ithin the stimulus materials. 

Reaching Unwarranted Conclusions or Incorrect Synthesis. Five exchanges 
were coded as incorrect synthesis in the observation notes. These occurred in 
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sessions one and three. Students who conunltted these errors labeled a problem 
"slaple" in the first session and moved on to other issues, made action 
decisions in the first session ("I would give parents the option to pull kids 
out [of a class]. '^), or reached conclusions that ignored other critical issues 
such as budget or policy ("Well, if all you need is an aide, couldn't the 
school just provide that?'^). 

The instructor' s participant observation notes provided data about ways 
this problem- solving error emerged. One student adopted a decision tree 
method for defining and attacking problems and always identified two branches 
to each level (either this or that). He prepared an elaborate chart of 
branching issues with bipolar extremes as the only options and persuaded 
another student to model this system during the third session. This system 
led to unwarranted conclusions, Lut it also encouraged the studant to ignore 
information that conflicted with his early judgments about the nature of the 
problems he confronted. The second student later abandoned the decision trees 
he had developed for more diver^^nt jtions before developing his plan. When 
final action plans were writteii and presented, one review panel member labeled 
the first students' recommendations excessively authoritarian, lacking in 
creative team or group solutions, and arrogant. 

The action plans also revealed this error more vividly than did 
observation notes. As students sought research, theory, or exemplary practice 
citations to support their conclusions, several found themselves stjnnied. 
Three of the five final action plans completed suffered at least in part from 
this error. 

Failing to Reach Warranted Conclusions or Inadequate Synthesis. This 
error show up only in later sessions and in action plans. Coded only three 
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times in observation notes, it was noted by the review panel in two of the 
five action plans. The two students who committed this error in their action 
plans were the two who were early in their administrative course work. One of 
the students phrased conclusions tentatively and provided overly lengthy 
explanations nd justifications for each action recommended. The other 
planned intrusive actions insupportable under state law and research on 
effective practice and ignored important actions warranted by the data. 

Premature Closure. Premature closure oc rred early (six incidents in 
session one, two in session three). The two students most likely to commit 
this error also suffered from incorrect and inadequate synthesis. Both had 
strong opinions about what "ought to be." One laughed during one point in the 
first session and said, "I don't see what the problem is. The principal 
should just cell tbe Leacher she has to do what he says." She didn't think 
one of the issues was even appropriate in the stimulus material because a 
principal should be able to demand compliance regardless of an external or 
professional justification for action. 

Selective Inattending or Anchoring, The two students who reached 
premature closure (in the first session) also failed to respond to information 
inconsistent with the thinking and experience they brought to the design 
studio. One remained steadfi st in her surety. One finally recognized, as 
noted above, that he had only tapped the "tip of the iceberg." This error 
also was difficult to detect in the observation notes, partly because the one 
student most likely to commit it failed to attend two of the four sessions, 
but partly because coaches didn't probe statements by students, helping to 
reveal the error. 
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Information and Answer Searchca bv Students 

Twelve times during the four sessions, students asked coaches directly 
what they would do. Often, coaches^ advice giving followed these requests. 
One student openly asked "^what do you think about this problems?^' during the 
first session. Another asked a coach, ''How realistic Is this?'' Student 
Insecurity led to other questions: "Have you had a problem like this? How 
did you respond to it?" At other times students asked for a summary judgment 
from the coaches, pressuring them to give an answer: "Is it best to go 
through the parents or wjrk it out with the legal counsel?" 

Students and coaches recognized this problem and talked about it in 
debriefing sessions and exit interviews. Lack of preparation for recognizing 
problem-solving errors and dealing with the coaches led them to recommend 
specific training to address this problem. 
Exit Interviews 

With reassuring frequency, students identified the same advantages and 
disadvantages to a design studio, problem-based approach. Students were 
asked: (a) Could you give me a general reaction to your experience with the 
design studio as part of an educational administration program? (b) What 
direction did coaches' questions take that was most helpful to you as a 
student? Did the most helpful type of questions change over the four 
sessions? (c) What kind of questions from coaches had the positive impact on 
your own problem-solving process? (d) What would you say might be the major 
advantages and disadvantages of an approach such as the design studio in 
educational administration progr^ins? (e) What did you like most and least 
about the stimulus material? What changes would you recommend? and (f) What 
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changes would you reconunend for future studio experiences that would make it a 
better learning experience for students? 

The gene^^l reaction of students to the design studio was positive. 
Students saf ''really enjoyed It/' found It to be a ''wonderful 

experience" o ,.ich they ''always looked forward to coming.'' One student 
labeled it his ''most exciting class so far." Another called it the "most 
difficult class but equally rewarding." The students argued that the studio 
made "other classes meaningful." The problem-based, interactive nature of the 
course formed the central feature on which students focused praise. They 
found they "liked mixing with other class members," "readir the materials," 
and interacting with the coaches. As a result, they argued that the studio 
gave them a broader "perspective of what a principal does" and "helped [them] 
look at a range of possibilities." The time pressure caused by the need to 
address five complex factual situations in a school and make action 
recommendations in six weeks made one student feel "against a wall and unclear 
about what [he] was doing." Another pointed out that they spent "a lot of 
time." 

Coaches helped students most "when they tried to get me away from my 
tunnel vision, when the students made the effort to bring more ideas with them 
to sessions, and when they r. .^arched for knowledge. One said, "The more ideas 
I had, the more they gave me," while another pointed out, "Questioning was 
more helpful as I did more reading." Students felt that questions that forced 
thero to "go broader" in early sessions and "narrow down" in the last sessions 
helped them the most. They also admitted that "at first, questioning was hard 
to deal with." 
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Feedbackp back and forth between coach and student, had the most 
positive impact ' their problem- solving , according to students. They like 
''what if and ''what about" questions and praised coaches who responded to 
their conclusions by asking, "If you did this, can you see that this might be 
a problem?" in later sessions. 

The advantages students identified centered around the studio' s 
"absolutely practical" approach to administration, the "controlled situation," 
and "working with coaches" to develop a stronger sense of their own approach 
to administration. They said it provided "insight into what it is really like 
to be a principal" and let them see there are "multiple methods to solving 
problems." One student argued that the questioning and problem-solving would 
be "helpful for future interviews" for administrative ,'obs and another that it 
provided "insight for future classes." Like the coach^ the students 
identified disadvantages that might be construed as contradictory to the 
advantages; while one student said the studio made her more sure of herself, 
another said made him less sure of himself. While one said it helped them 
see that multiple solutions are possible, another identified the fact that he 
"wondered if there is an ultimate solution" as a disadvantage. However, 
students d'^d say that the complexity of problems- -five situations, each with 
many issues- -was overwhelming at times and felt the time pressures were not 
realistic. 

Students liked most the realism, high interest, and challenge of 
confronting the issues in the stimulus materials. They liked lease the 
complexity of overlapping problems, a lack of skills on their part for linking 
theory to the action plans, and how draining the pressure to produce and 
defend an action plan was for thera. 
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The changes recommended by students also mirror recommendations from 
coaches and illustrate the tension between creating a manageable problem with 
enough complexity to convey reality in administrative work* Students wanted 
some time to work (perhaps a two week break in the middle) without Interacting 
with coaches so they would have more to present. They also would have liked a 
more lengthy presentation and defense time. All these requests place 
additional pressure on the time frame of an academic schedule. Several felt 
that assigned readings left them with too much to absorb prior to the coaching 
sessions . 

Summary and Conclusions 
The foregoing findings lead to a number of general conclusions. Coaches 
pointed out ways in which they had grown, even wishing they had been able to 
have a similar experience during their own professional educations, and ways 
the studio could be used for inservice professional development. SLu.snts 
praised the chance to engage one-v^n-one in problem- solving with thoughtful, 
expert practitioners, the direct application of knowledge to action, and the 
focus on their own responses to the problems of practice. They also 
identified forms of questioning they felt facilitated their own thinking and 
helped them move toward planning and justifying their actions. Areas that 
needed further development became apparent from the observations, interviews, 
and action plans. The writing and sequencing of the stimulus materials, 
training and e^cperience of the cognitive coaches, structure of the coaching 
sessions (including forms of questioning deliberately sequenced in the planned 
sessions), student response formats and action plans, and subject-based 
knowledge foundation of students and coaches all raised questions. A number 
of more specific conclusions emerged from the analysis as well. 
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First, coaches require training and practice in reflective and inductive 
questioning. In order to help students define problems, assess what is known, 
survey possible actions, and make action plan decisions, coaches need personal 
expertise and experience in administration but also in the cognitiv^e processes 
on which the design studio depends. This takes resources, effort, and time 
and makes it unlikely that a design studio program can function using new 
coaches for each studio or without resources to support the coaches. By using 
technology fuch as interactive video disk, this high human resource demand 
could be alliwiated, but the personal nature of interactions between coaches 
and students provided a signature feature of the design studio frequent-, 
praised by participants. We believe it should not be abandoned. 

Second, productive questioning techniques at the beginning of problem 
definition differ from the most effective questioning and interaction 
techniques near the end of a studio experience. When students begin to probe 
the stimulus cases, they often f )cus on issues familiar to them or apply 
inappropriate and limited personal experiences, reaching premature conclusions 
or ignoring diverse possibilities. Questions at the beginning of problem 
definition that stimulate divergent thinking and highlight that students ''do 
not know what they do not know" stimulate quick engagement and increase search 
for knowledge at the early problem-solving stages. Questions that help 
students reach closure by eliminating options and explaining rationales lead 
to more singular and concrete action plans toward the end of the studio 
experience. 

Third, serious issues related to the stage in a preservice program 
during which a design studio mi^ht be most useful require further examination. 
While students near the end of their course work argued that they needed 
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problem-based experiences earlier in order to become more skilled in problem- 
solving techniques that involve multiple issues and to develop explicit l^nks 
between what they were learning and what they might do, those nearer the 
beginning of formal course work felt restricted by their limited subject- 
based knowledge. The subject-based study of the foundation readings in four 
weeks left students reeling over too much too fast and reporting significant 
cognitive overload. Yet, without these readings, students faced the problem 
situations from radically uneven knowledge bases. These findings support the 
argument that problem-based learning can be built into a number of subject - 
centered courses more systematically and advantageously. However, a student- 
focused design studio requiring that students address a multifaceted and 
complex set of problems in unified ways, applying knowledge across a broad 
spectrum, should have fixed prerequisites to protect students. 

A fourth conclusion follows naturally from the third. Problem-based, 
student- centered learning may deserve examination hroughout professional 
preparation for educational administration. The experiences of a few 
professional schools that have reorganized their curriculums and programs of 
instruction around problem-based, student -centered methods support this 
conclusion (See for example, the Southern Illinois and Harvard University 
medical schools and the doctorate of pharmacy program. University of Utah). 

Fifth, the preparation of good teaching cases requires more than the 
compilation of real experiences from the field. Depending on the stage of a 
program of study and the learning goals of the course or field experience, a 
teaching case may need to be more or less complex and interactive, point 
students directly toward different reference materials or knowledge bases, and 
raise dilemmas of varying familiarity. This need could be met by 
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systematically collecting vivid cases arising In school administration as part 
of the knowledge base (Osterman, 1989) and by developing cases designed to 
teach specific subjects as part of the formal curriculum for educational 
administration education. 

Slxthp students in educational administration experience problem- 
solving successes and errors similar to those encountered by people In other 
situations. They learn to seek solutions through diagnostic questioning, 
synthesis, brainstorming, and the application of experience and knowledge. 
They make some common errors --seeking data that will not be helpful; reaching 
unwar^r^nted conclusions; failing to reach conclusions warranted by the data; 
premature closure; and selective inattending or failure to attend to new 
information which seems inconsistent with earlier thinking. These successes 
and errors occur with different frequency at successive stages of the problem- 
solving process. Early in the process, the students were more likely to rely 
on more familiar problem-solving techniques, such as brainstorming and testing 
coaches^ responses to their ideas. Only two students developed proficiency at 
relevant diagnostic questioning. Early errors also differ from those common 
in the later stages. In the first two sessions, the students in our study 
were more likely to reach unwarranted conclusions, reach premature closure, 
and fai to attend to new information Inconsistent with their earlier 
thinking. Later on, students more frequently failed to reach warranted 
conclusions and reached unwarranted conclusions while finding it difficult to 
make explicit connections between their knowledge and experience and the 
actions they recommended. Certain kinds of errors are more common for some 
students than for others, suggesting that students need careful diagnosis of 
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their reflection and assistance with that process during the studio 
experience , 

Several problems arose In recognizing problem- solving errors during 
coaching E.^f^sicns and In coding problem- solving errors during analysis. 
First, the cuvagulty In the school-wide situations depicted In the stimulus 
materials, which was necessary to maintain a balance between complexity and 
manageability, made coaches and researchers reluctant to dismiss the relevance 
of any Information search In early stages of the design studio. Only 
blatantly disconnected searches clearly warranted this label. Second, the 
Indeterminacy of administrative work made Incorrect synthesis and Inadequate 
synthesis difficult to Identify. As there always are a number of possible 
problem definitions and solutions that might be warranted, these errors were 
not easy to decipher during sessions and only became apparent when final 
action plans were designed. Two of the five students reached a number of 
conclusions that review panel members felt were unwarranted or Insufficiently 
supported by the students. One (who was at the beginning of her 
administration course work) made inadequate recommendations, falling to bring 
her problem definition and solution process to adequate closure according to 
the review panel. Two of the five (one elementary and one secondary) 
presented action plans that the review panel found creative, appropriate, and 
complete. Third, premature closure and anchoring (lack of attention to new 
Information inconsistent with earlier thinking) occurred with more frequency 
(eight and four coded references respectively) and early on In sessions. 
Anchoring also could be Identified In the final action plans. 

Coaches need to be explicitly prepared to recognize and deal with common 
errors and have strategies and techniques to point students away from them. 
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We found them difficult to differentiate in early stages of the studio, in 
part because no advance work with the specific combination of challenges in 
the stimulus materials was undertaken and no practice identifying problem- 
solving errors was provided. Coaches found that general reading and brief 
training in reflective practice techniques and cognition research was 
insufficient preparation to deal systematically with these problems. This 
finding raises the possibility that training for coaches, active engagement by 
coaches throughout the quarter with the instructor, and final assessment 
should be increased substantially. This will require more resources and more 
commitment from coaches who are already busy administrators. 

In conclusion, while much talk abounds about the importance of change 
leadership and problem solving for school administrators, little systematic 
alteration of professional preparation programs is accompanying school reform 
(Hart, 1990b). This study reports some results of an attempt to make such a 
change grounded in cognitive theories of adult learning and action. The 
results present a complex yet realistic view of changes in professional 
preparation that can enhance parallel attempts to improve educational practice 
and leadership. It provides options for reform internally (within courses 
currently taught) and systemically (of entire cognitive structures undei lying 
degree programs). 
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